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METHOD FOR INTERACTIVELY VIEWING FULL-SURROUND IMAGE DATA 

AND APPARATUS THEREFOR 

BACKGROUND OF THE INVENTION 

The present invention relates generally to a method and corresponding apparatus for viewing 
images. More specifically, the present invention relates to a method and corresponding apparatus 
for viewing full-surround, e.g., spherical, image data. 

Systems and techniques for changing the perspective of a visible image in producing a 
resultant image, or systems and methods of transforming an image from one perspective form to 
another have been the subject of scientific thought and research for many years. Systems and 
techniques for transforming visible images can generally be divided into three separate categories: 

(1) perspective generation systems and methods suitable for applications such as flight 
simulators; 

(2) three-dimensional (3D) to two-dimensional (2D) conversion systems and methods; and 

(3) miscellaneous systems and methods. 

The first category includes U.S. Patent No. 3,725,563, which discloses a method of and 
apparatus for raster scan transformations using rectangular coordinates which are suitable for 
electronically generating images for flight simulators and the like. More specifically, the patent 
discloses a technique for raster shaping, whereby an image containing information from one 
viewpoint is transformed to a simulated image from another viewpoint. On the other hand, U.S. 
Patent No. 4,763,280 discloses a curvilinear dynamic image generation system for projecting 
rectangular coordinate images onto a spherical display surface. In the disclosed system, rectangular 
coordinates are converted to spherical coordinates and then the spherical coordinates are distorted 
for accomplishing the desired simulation of curvature. 
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The second category of systems and techniques perform 3D-to-2D conversion, or vice versa. 
For example, U.S. Patent No. 4,821,209 discloses a method of and apparatus for data transformation 
and clipping in a graphic display system, wherein data transformation is accomplished by matrix 
multiplication. On the other hand, U.S. Patent No. 4,667,236 discloses a television perspective 
5 effects system for providing perspective proj ection whereby each point of a three-dimensional obj ect 
is projected onto a two-dimensional plane. New coordinates X f and Y ? are prepared from the original 
coordinates X, Y and Z, and the viewing distance D, using the general formulas X -XD/Z and 
Y -YD/Z. As the object to be displayed is rotated around the X or Y axis, the viewing distance D 
is changed for each point. 

10 

m In the third category, miscellaneous systems and methods are disclosed by, for example, U.S. 

] j! Patent No. 5,027,287, which describes a device for the digital processing of images to obtain special 
III geometrical effects wherein digital image data corresponding to intersection points on a rectangular 
Cj X, Y grid are transposed by interpolation with respect to intersection points of a curved surface. U.S. 
1 1; Patent No. 4,882,679, on the other hand, discloses a system and associated method of reformatting 
images for three-dimensional display. The disclosed system is particularly useful for generating 
three-dimensional images from data generated by diagnostic equipment, such as magnetic resonance 
imaging. 

20 However, none of the above described methods or systems permit viewing in circular 

perspective, which is the best way to view spherical data. It does all that linear perspective does 
when zoomed in, but it allows the view to zoom out to the point where the viewer can see almost 
everything in the spherical data simultaneously in a visually palatable and coherent way. 

25 What is needed is a method for viewing full-surround, e.g., spherical, image data employing 

circular perspective. Moreover, what is needed is an apparatus for viewing full-surround, e.g., 
spherical, image data employing circular perspective. What is also needed is a method for viewing 
full-surround, e.g., spherical, image data employing circular perspective which is computationally 
simple. Preferably, the method for viewing full-surround, e.g., spherical, image data employing 
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circular perspective can be employed on any personal computer (PC) system possessing a three 
dimensional (3-D) graphics capability. 

SUMMARY OF THE INVENTION 

Based on the above and foregoing, it can be appreciated that there presently exists a need in 
the art for a viewing methods and corresponding apparatuses which overcome the above-described 
deficiencies. The present invention was motivated by a desire to overcome the drawbacks and 
shortcomings of the presently available technology, and thereby fulfill this need in the art. 

The present invention implements a novel and practical circular perspective viewer for 
spherical data. Moreover, it implements the circular perspective viewer within the context of 
existing 3D graphics utilities native to personal computers (PCs). Thus, the method and 
corresponding apparatus for circular perspective viewing is practical for a broad market. 

One object according to the present invention is to provide a method and corresponding 
apparatus for modeling the visible world by texture mapping full-surround image data. 

Another object according to the present invention is to provide a method and corresponding 
apparatus for modeling the visible world by texture mapping full-surround image data onto a 
p-surface whereby the resultant texture map is substantially equivalent to projecting full-surround 
image data onto the p-surface from a point Q inside the region X of the p-surface. 

Still another object according to the present invention is to provide a method and 
corresponding apparatus for modeling the visible world by texture mapping full-surround image data 
wherein the viewer is allowed to interactively rotate the model. 

Yet another object according to the present invention is to provide a method and 
corresponding apparatus for modeling the visible world by texture mapping full-surround image data 
wherein the viewer is allowed to interactively change the direction of vision. 



A still further object according to the present invention is to provide a method and 
corresponding apparatus for modeling the visible world by texture mapping full-surround image data, 
wherein the viewer is allowed to interactively alter the focal length or view angle. 

Another object according to the present invention is to provide a method and corresponding 
apparatus for modeling the visible world by texture mapping full-surround image data, wherein the 
viewer is allowed to interactively move the viewpoint. 

Still another object according to the present invention is to provide a method and 
corresponding apparatus for modeling the visible world by texture mapping full-surround image data, 
wherein the viewpoint is close to the surface of the p-sphere. 

Another object according to the present invention is to provide a method and corresponding 
apparatus for modeling the visible world by texture mapping full-surround image data, wherein the 
viewer is allowed to interactively move the viewpoint. 

A further object according to the present invention is to provide a method and corresponding 
apparatus for modeling the visible world by texture mapping full-surround image data, wherein the 
viewer is allowed to select an area of the image and cause another model of the visible world to be 
loaded into said viewing system. 

Another object according to the present invention is to provide a method and corresponding 
apparatus for modeling the visible world by texture mapping full-surround image data, wherein the 
viewer is allowed to perform any combination of actions specified immediately above. 

It will be appreciated that none of the above-identified objects need actually be present in 
invention defined by the appended claims. In other words, only certain, and not all, objects of the 
invention have been specifically described above. Numerous other objects advantageously may be 
provided by the invention, as defined in the appended claims, without departing from the spirit and 



scope of the invention. 



These and other objects, features and advantages according to the present invention are 
provided by a method of modeling of the visible world using full-surround image data. Preferably, 
the method includes steps for selecting a view point within a p-surface, and texture mapping 
full-surround image data onto the p-surface such that the resultant texture map is substantially 
equivalent to projecting fall-surround image data onto the p-surface from the view point to thereby 
generate a texture mapped p-surface. 

According to one aspect of the invention, the method also includes a step for either rotating 
the texture mapped p-surface or changing the direction of view to thereby expose a new portion of 
the texture mapped p-surface. According to another aspect of the invention, a first the texture 
mapped p-sphere is replaced by a second texture mapped p-sphere by interactively selecting the new 
viewpoint from viewpoints within the second texture mapped p-sphere. 

These and other objects, features and advantages according to the present invention are 
provided by a method of modeling of the visible world using fall-surround image data, the method 
comprising steps for providing the fall surround image data, selecting a view point within a p- 
surface, texture mapping fall-surround image data onto the p-surface such that the resultant texture 
map is substantially equivalent to projecting fall-surround image data onto the p-surface from the 
view point to thereby generate a texture mapped p-surface, and displaying a predetermined portion 
of the texture mapped p-sphere. 

These and other objects, features and advantages according to the present invention are 
provided by an apparatus for modeling the visible world using full-surround image data, comprising 
first circuitry for selecting a view point within a p-surface, second circuitry for texture mapping 
fall-surround image data onto the p-surface such that the resultant texture map is substantially 
equivalent to projecting fall-surround image data onto the p-surface from the view point to thereby 
generate a texture mapped p-surface, and third circuitry for displaying a predetermined portion of 



the texture mapped p-sphere. 

These and other objects, features and advantages of the invention are disclosed in or will be 
apparent from the following description of preferred embodiments. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and various other features and aspects of the present invention will be readily 
understood with reference to the following detailed description taken in conjunction with the 
accompanying drawings, in which like or similar numbers are used throughout, and in which: 

Fig. 1 illustrates a set of all rays from a predetermined viewpoint, which illustration facilitates 
an understanding of the present invention; 

Fig, 2 illustrates a set of points, excluding the viewpoint, located on a corresponding one of 
the rays, which illustration facilitates an understanding of the present invention; 

Fig. 3 illustrates the formation of a projection of the set of points, or a subset thereof, 
illustrated in Fig. 2; 

Figs. 4A and 4B illustrate the resultant images generated by two different projections, 
respectively, given a constant viewpoint; 

Fig. 5 illustrates the concept of linear perspective: 
Fig. 6 illustrates the concept of circular perspective; 
Fig. 7 illustrates the concept of stereographic projection; 

Fig. 8 is a high level block diagram of a circular perspective viewing system according to the 
present invention; 

Figs. 9A through 9G collectively form a listing of the dedicated code for converting a general 
purpose computer system into the circular perspective viewing system illustrated in Fig. 8; and 

Figs. 1 OA and 1 OB collectively forming a listing of an exemplary code block for triangulating 
a hemisphere and texture coordinates. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The method and corresponding apparatus according to the present invention are similar to 



that disclosed in U. S. Patent No. 5,684,937, which patent is incorporated herein by reference for 
all purposes, in that it generates perspective views derived for constants less than or equal to two, 
and greater than or equal to one, i.e., 1.0 < X < 2.0. However, it will be appreciated that the 
inventive method and apparatus are different from U. S. Patent No. 5,684,937 in that the method for 
5 deriving perspective is different than 'explicitly' dividing all angles by a selected constant, as 
disclosed in that patent. Instead, the angles are 'implicitly 1 divided by a constant by moving the 
viewpoint around inside a "p-sphere". Additional details will be provided below. 

By employing the method and corresponding apparatus according to the present invention, 

10 it is possible to create a virtual pictosphere using a conventional 3-D graphics system. Preferably, 

,^ the inventive method an apparatus texture map the visible world onto a sphere. It should be 

0 mentioned that when the user selects a viewpoint at the center of this sphere and renders the view 

ry using the primitives of a conventional 3D graphics system, the used implicitly divides all angles by 

^ one, and result a linear perspective view. However, when the user selects a viewpoint on the surface 

if; of this sphere, selects a direction of view towards the center, and renders the view using the 

- primitives of a conventional 3D graphics system, the user implicitly divides all angles by two, thus 

£7 creating a circular perspective view. Moreover, by allowing the viewpoint to move around within 

^ or on the sphere, the user achieves results virtually identical to those achieved by U. S. Patent No. 

"i U 

: § 5,684,937 for constants ranging from 1 .0 to 2.0. 

20 

It will be appreciated that the method and corresponding apparatus according to the present 
invention implement a novel and practical circular perspective viewer of spherical data. The 
inventive method and apparatus advantageously can be achieved within the context of existing 3-D 
graphics utilities and hardware native to PCs. It will be noted from the statement immediately above 
25 that the inventive method and apparatus advantageously can be implemented in a broad range of 
existing systems. 

The method and corresponding apparatus according to the present invention are predicated 
on the following starting, i.e., given, conditions: 
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(1) the set of all rays V from a given point VP, as illustrated in Fig. 1 ; 

(2) a set of points P not including VP, each point in P being contained by one and only 
one ray in V, as illustrated in Fig. 2; and 

(3) the set of color values C, each color in C being associated with one and only one ray 
in V, and also thereby associated with the point in P contained by said ray. 

Moreover, the following definitions apply: 

(1) POINTS P: The visible world. 

(2) A PROJECTION OF P: A subset of points P. Any number of points Pn contained 
in P may be slid closer to or further from point VP along their corresponding rays. 
The resultant new configuration of points P is called a projection of P. The concept 
can best be understood by referring to Fig. 3; 

(3) MAGIC POINT, VIEWPOINT, OR POINT OF PROJECTION: Point VP. Please 
note, no matter how points P are projected, their appearance will remain the same 
when viewed from point VP. This latter concept may best be understood by referring 
to Figs. 4A and 4B. 

(4) FULL-SURROUND IMAGE DATA: data which samples the points P. This data 
encodes, explicitly or implicitly, the association of a color value with a given 
direction from a given point of projection. It should be mentioned that this point that 
full-surround image data is useful in many fields of entertainment because, when 
delivered to many viewers, it enables the construction of an independent viewing 
system defined below. 

(5) P-SPHERE: a computer graphics representation of any polyhedron where there exists 
at least one point x inside (neither intersecting, nor lying outside) the polyhedron 
which may be connected to every point of the polyhedron with a distinct line 
segment, no portion of which said line segment lies outside the polyhedron or 
intersects the polyhedron at a point not an endpoint. The union of all such points x 
form the region X of the p-sphere. For a convex p-sphere, the region X is all points 



of the interior of the p-sphere. Examples of computer graphics objects which may 
be modeled as p-spheres include a tetrahedron, a cube, a dodecahedron, and a faceted 
sphere. 

P-SURFACE: a computer graphics representation of any surface with a well-defined 
inside and outside, where there exists at least one point x inside (neither intersecting, 
nor lying outside) the surface which may be connected to every point of the surface 
with a distinct line segment, no portion of which said line segment lies outside the 
surface or intersects the surface at a point not an endpoint The union of all such 
points x form the region X of the p-surface. For a convex p-surface, the region X is 
all points of the interior of the p-surface. Examples of computer graphics objects 
which may be modeled as p-surfaces: tetrahedron, cube, sphere, ellipsoid, cylinder, 
apple torus, lemon torus, b-spline surfaces closed or periodic in u and v. A p-sphere 
is a p-surface. 

LINEAR PERSPECTIVE: the projection of a portion of the visible world onto a 
plane or portion of a plane, as illustrated in Fig. 5. 

CIRCULAR PERSPECTIVE: the projection of the visible world, or portion thereof 
onto a plane, or a portion of a plane, after performing the perspective transformation 
of the visible world according to patent 5,684,937, where the constant used is 2. 
After such a transformation, when the direction of vision specified in said 
transformation is perpendicular to the proj ection plane, there is a one-to-one mapping 
of all the points P defining the visible world and all the points of an infinite plane. 
This definition is illustrated in Fig. 6; 

STEREOGRAPHIC PROJECTION: the one-to-one mapping of each point on a 
sphere to each point of an infinite tangent plane, said mapping performed by 
constructing the set of all rays from the antipodal point of the tangent point, the 
intersection of said rays with the plane defining said mapping. The understanding 
of this definition will be facilitated by reference to Fig. 7. Please note that circular 
perspective and stereographic projection produce geometrically similar (identically 
proportioned) mappings when the direction of vision specified in the perspective 



transformation of circular perspective contains the tangent point specified in 
stereographic projection; 

(10) INDEPENDENT VIEWING SYSTEM: an interactive viewing system in which 
multiple viewers can freely, independently of one another, and independently of the 
source of the image data, pan that image data in all directions with the effect that 
each viewer feels like they are "inside" of that imagery, or present at the location 
from which the imagery was produced, recorded, or transmitted; and 

(1 1) STANDARD COMPUTER GRAPHICS SYSTEM: a computer graphics system 
which supports linear perspective viewing, including the changing of the focal length 
or the altering of the view angle, the apparent rotation of viewed objects, and/or the 
apparent changing of direction of vision, and the texture mapping of image data onto 
objects within the class of p-surface. 

It will be appreciated that in a standard computer graphics system by texture mapping 
full-surround image data onto a p-surface such that the resultant texture map is effectively equivalent 
to projecting the full-surround imagery onto the p-surface from a some point Q contained in the 
region X of the p-surface, a representation of the visible world is achieved. 

Referring to Figs. 9A through 9G, the method for viewing full-surround, e.g., spherical, 
image data will now be described. It should be mentioned that the corresponding code implementing 
the inventive method is written in the "C" language, although a plurality of programming languages 
are well known and readily adapted to this purpose. It will also be appreciated that code lines 
starting with "gl M or "glut" indicate calls to a conventional graphics library (GL) such as OpenGL™. 
One of ordinary skill in the art will readily appreciate that the last function in this listing is called 
main(). This is where the program starts. 

It will be appreciated from inspection of Figs. 9A through 9G that the routine mainO calls 
various glut... functions to set up GL, the graphics library used here. It will be noted that the glut... 
functions are part of GL. Furthermore, it will be appreciated that main() registers or maps certain 
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keyboard events with GL with glutKeyboardFunc(Key). In other words, glutKeyboardFunc(Key) 
defines the response of the inventive method to operation of the corresponding "Key/ 1 

Moreover, Key() is a function describing the actions of GL to keyboard events. Of 
5 importance are keyboard events ( f z' and *Z f ) which move the viewpoint in and out along the Z axis 
relative to the "p-sphere", effectively altering the perspective of the view, and keyboard events (55, 
57, 52, 54, 56 and 50) which control rotation of the "p-sphere", effectively allowing the viewer to 
"look around". 

1% It will be noted that main() registers the function displayO with GL, with the 

* glutDisplayFunc(display) function. Moreover, display() uses the global variables controlled by 
11J KeyO to move the viewpoint along the Z axis relative to the "p-sphere", and to rotate the "p-sphere" 
5 relative to a constant direction of view. 

15; Preferably, display() builds the "p-sphere" with glCallList(current_texture->texl) and 

glCallList(current_texture->tex2). In the first instance, texl is mapped to a triangulation 
'Si approximating a hemisphere, and is added to the display list. In the second instance, tex2 is mapped 
S to the same hemisphere - after rotating it 1 80 degrees to form a sphere - and is also added to the 
display list, in the function readTexture(). Preferably, texl and tex2 are texture maps built from two 
20 pictures, respectively, taken with a fisheye lens. Advantageously, texl and tex2 collectively 
comprise a "pictosphere." It should be noted that the triangulation approximating the hemisphere 
was built in the function createHemisphere(), the full listing of which is found in Figs. 1 OA and 1 OB 
and not Figs. 9A - 9G. 

25 At this point, Key() and display() have been registered with GL. The code main() then calls 

initialize_objects() which actually calls the routines readTexture() and createHemisphereO. All the 
other functions are support functions. 
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It will be appreciated that the user now has an instance of a p-sphere in GL made by mapping 
two fisheye images, e.g., photographs, to two adjoining hemispheres to thereby generate full- 
surround, e.g., spherical, image data. The user advantageously can interactively move the viewpoint 
away from the center of the p-sphere and, if so desired, very near the inside surface of the p-sphere. 
It should be mentioned at this point that the direction of view is still towards the center of the 
p-sphere. Moving the viewpoint from the center of the p-sphere automatically generates a circular 
perspective view, which advantageously can be displayed on display screen 20 of the PC illustrated 
in Fig. 8. Moving back to the center of the p-sphere, permits the user to generate a linear perspective 
view. It will also be appreciated from the discussion above that it is possible to rotate the surface 
of p-sphere, thus simulating looking around within the p-sphere. 

It should be mentioned that by setting the viewpoint of the graphics system close to the center 
point of the p-sphere point and then enabling the viewer to rotate that p-sphere around a point close 
to the center point of the p-sphere, an independent viewing system providing linear perspective is 
achieved. Moreover, by adding the further capability of altering the focal length or angle of view, 
a zoom ability advantageously can be provided for the user. 

It should also be mentioned that in the case where the p-surface used to model the visible 
world is a good approximation of a sphere, that is, a substantially better model than a tetrahedron or 
a cube, and where the view point of that representation is close to the approximate center of that 
p-surface, then by allowing the viewer to move the viewpoint away from center point to a point close 
to the surface of the p-surface, an independent viewing system is achieved in circular perspective. 
This is astounding when one considers that the native graphics system of a conventional PC only 
supports viewing in linear perspective. The method and correspond apparatus according to the 
present invention work because such an independent viewing system models stereographic 
projection, which is geometrically similar to circular perspective. 

Furthermore, by letting the viewer move the viewpoint outside of the p-surface, the viewer 
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can get a feeling for how the independent viewing works. This can be useful for designers of 
systems containing many hyper-linked full-surround surfaces. For example, many p-spheres 
picturing the penthouse terraces of New York advantageously can be linked together so that the 
viewer may hop from p-sphere to p-sphere, simulating a tour of the terraces. 

5 

The above described method of the invention may be performed, for example, by the 
apparatus shown in FIG. 8. This viewing apparatus is composed of a central processing unit (CPU) 
1 0 for controlling the components of the system in accordance with a control program stored in read- 
only memory (ROM) 60 or the like. The CPU 1 0 stores temporary data used during execution of the 
l<3k inventive method, i.e., viewing method, in random-access memory (RAM) 40. After the majority 
jO of the method steps are performed, the generated visible points are displayed on display device 20 
nj (e.g., cathode ray tube (CRT) or liquid crystal display (LCD)) as visible points in accordance with 
'Q the appropriate color values stored in color table 30, e.g., a color lookup table (CLUT) found in the 
■1! graphics controller in most PCs. Advantageously, a spherical data generator device 70, such as a 
IS camera or the like, and preferably the data generator system disclosed in pending application serial 
;2 No. 08/749, 1 66 (filed November 1 4, 1 996; allowed May 6, 1 998), which application is incorporated 
herein by reference for all purposes, may be used to generate different color values corresponding 
# to the visible points of a viewed object, image or picture. An input device 50 is provided for entering 
data such as the viewer's viewing direction, reference plane, configuration (scaling) factor k, and 
20 other pertinent information used in the inventive viewing method. 

As mentioned above, it will be appreciated that the method and corresponding apparatus 
according to the present invention advantageously can be used in an audiovisual or multimedia 
entertainment system. In particular, since each user advantageously can select his or her preferred 
25 view point and direction of view, multiple user can receive a single set of full-surround image data 
and generate corresponding multiple display images, in either linear or circular perspective. 

Although presently preferred embodiments of the present invention have been described in 

-13- 
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detail hereinabove, it should be clearly understood that many variations and/or modifications of the 
basic inventive concepts herein taught, which may appear to those skilled in the pertinent art, will 
still fall within the spirit and scope of the present invention, as defined in the appended claims. 
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WHAT IS CLAIMED IS: 

1. A method of modeling of the visible world using fall-surround image data, said method 
comprising steps for: 

selecting a view point within a p-surface; and 

texture mapping full-surround image data onto said p-surface such that the resultant texture 
map is substantially equivalent to projecting fall-surround image data onto the p-surface from said 
view point to thereby generate a texture mapped p-surface. 

2. The method as recited in claim 1, further comprising the step of rotating said texture 
mapped p-surface so as to simulate rotating the direction of view in the opposite direction. 

3. The method as recited in claim 1 , wherein said selecting step comprises selecting the view 
point and a direction of view, and wherein said method further comprises the step of interactively 
changing said direction of view to thereby expose a corresponding portion of said texture mapped 
p-surface. 

4. The method as recited in claim 1, farther comprising the step of displaying a 
predetermined portion of said texture mapped p-surface. 

5. The method as recited in claim 4, wherein the viewer is allowed to interactively alter at 
least one of focal length or an angle of view relative said textured mapped p-surface to thereby vary 
the displayed portion of said texture mapped p-surface. 

6. The method as recited in claim 1, farther comprising the steps of: 
selecting a new viewpoint; and 

repeating said texture mapping step using said new viewpoint. 
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7. The method as recited in claim 6, wherein the resultant viewpoint is close to the surface 
of said p-sphere. 

8. The method as recited in claim 6, wherein said selecting step comprises interactively 
selectively said new viewpoint. 

9. The method as recited in claim 8, wherein a first said texture mapped p-sphere is replaced 
by a second texture mapped p-sphere by interactively selecting said new viewpoint from viewpoints 
within said second texture mapped p-sphere. 

10. A method of modeling of the visible world using full-surround image data, said method 
comprising steps for: 

providing said full surround image data; 
selecting a view point within a p-surface; 

texture mapping full-surround image data onto said p-surface such that the resultant texture 
map is substantially equivalent to projecting full-surround image data onto the p-surface from said 
view point to thereby generate a texture mapped p-surface; and 

displaying a predetermined portion of said texture mapped p-sphere. 

11. The method as recited in claim 10, further comprising the step of rotating said texture 
mapped p-surface so as to simulate rotating the direction of view in the opposite direction. 

12. The method as recited in claim 10, wherein said selecting step comprises selecting the 
view point and a direction of view, and wherein said method further comprises the step of 
interactively changing said direction of view to thereby display another portion of said texture 
mapped p-surface. 

13. The method as recited in claim 10, further comprising the steps of: 
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selecting a new viewpoint; and 

repeating said texture mapping step using said new viewpoint. 

14. The method as recited in claim 13, wherein said selecting step comprises interactively 
selectively said new viewpoint. 

15. The method as recited in claim 13, wherein a first said texture mapped p-sphere is 
replaced by a second texture mapped p-sphere by interactively selecting said new viewpoint from 
viewpoints within said second texture mapped p-sphere. 

1 6. An apparatus for modeling the visible world using full-surround image data, comprising: 
means for selecting a view point within a p-surface; 

means for texture mapping full-surround image data onto said p-surface such that the 
resultant texture map is substantially equivalent to projecting Ml-surround image data onto the 
p-surface from said view point to thereby generate a texture mapped p-surface; and 

means for displaying a predetermined portion of said texture mapped p-sphere. 

17. The apparatus as recited in claim 16, wherein said selecting means comprises means for 
selecting said view point and interactively selecting a direction of view to thereby interactively 
display portions of said texture mapped p-surface. 

18. The apparatus as recited in claim 16, wherein said selecting means provides for 
interactive selection of said viewpoint. 

1 9. The apparatus as recited in claim 1 7, further comprising means for replacing a first said 
texture mapped p-sphere by a second texture mapped p-sphere by interactively selecting said 
viewpoint from a plurality of viewpoints within said second texture mapped p-sphere. 
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ABSTRACT OF THE DISCLOSURE 

A method of modeling of the visible world using full-surround image data includes steps for 
selecting a view point within a p-surface, and texture mapping full-surround image data onto the 
p-surface such that the resultant texture map is substantially equivalent to projecting full-surround 
image data onto the p-surface from the view point to thereby generate a texture mapped p-surface. 
According to one aspect of the invention, the method also includes a step for either rotating the 
texture mapped p-surface or changing the direction of view to thereby expose a new portion of the 
texture mapped p-surface. According to another aspect of the invention, a first the texture mapped 
p-sphere is replaced by a second texture mapped p-sphere by interactively selecting the new 
viewpoint from viewpoints within the second texture mapped p-sphere. A corresponding apparatus 
is also described. 
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/* Includes required 
# include <GL/gl.h> 
8 include <GL/glut.h> 

include <stdio.h> 
if include <ppm.h> 
8 include <ntath.h> 



* something because of windows 
•/ 

void eprintfO { 



* cur data structure of choice pjg 



typedef struct obj t 

;* other parameters */ 
float matrix 1 16) ,* 



/* view angle V 
float viewangle; 

/* aspect ratio */ 
float aspect; 

/* z of the camera */ 
float tz; 



/* ry of the camera */ 
float ry; 
> Obj; 

/* hold the display lists for textures »/ 
typedef struct texture { 

int texl; 

int tex2? 
) Texture; 

/** 

* our global variables 
*/ 

/* camera settings ■/ 
Obj scene; 

/* texture stuff */ 
Texture def; 

Texture* current_texture - &de£; 

/* track the next display list number */ 
int next DLnuift = 2; 



/* stuff for lighting */ 

float HghtPos[4] = (2.0, 4.0, 2.0, 0}; 

float UghtDirMl - CO. D, 1.0. 1.0}? 

float lightAmb[4) = (0.4, 0,4, 0.4, 1,31? 

float ligbtDif£[4] = {0.8, 0.3, 0.6, 1.0); 

float lightSpec[4) = (0.8, 0.3, 0.8, 1-0}; 

j.nt lights = G; 

int outsideView - 0j 

int parent; 



^define HEMISPHERE ; 

void createHemisphereUnt listNura, int numPts, int geomi; 



* Read in the ppm files and create display lists for a texture 

* returns the dimension of the image 
*/ 

pixel **reapl, **map2? 

GLufcyce *texl, *tex2, **tmpPP, *tmpPj 

void readTexture (Texture* t, char* filel, char* file2) { 
FILE *fpl, *fp2; 
int cols, rows, i, j, index; 
pixval maxval? 

/* open the files */ 
fpl = fopenffilel, -r"> ; 

fp2 = fopen<file2, -r">; Fig, 9B 

if (ifpl) { 

fprintf <stderr, "Couldn't open %s\n*, filelJr 

) 

if (!fp2) { 

fprintf <stderr t -Couldn't open %s\n", file2*; 

] 

J* read the ppm files */ 

inapl = ppm_readppm ( £pl r icols, trows, Smaxvall; 

fprintf (stderr, 'fcs: rows = ftd U cols = %d\n", filel, rows, cols, maxval]; 
map2 = ppnureadppm f Ep2 , ficols, *rows # bmaxval}; 

fprintf (stderr, 'fcs: rows = fed U cols = %d\n', file2, rows, cols, maxval); 
)+ convert them */ 

texl - raalioc {sizeof (GLubyte) * rows * cols * 3); 
tex2 = mallocisizeof (GLubyte) * rows * cols * 3); 
index = 0; 

for (i = Of i < rows; i++) { 

for (j = 0; j < cots; { 
/* R V 
texl ! index] 
tex2 [index] 
index ++j 

/* G */ 
zexl [ index] 
zex2 [index] 
index +4-; 

/* B */ 
texl [index] 
cex2 [index) 
index ++; 

} 

} 

/* create the textures */ 

/* new display list*/ 

glNewList (nextDLnum, GL_COMPILEy ; 

t->texl = nextDLnum; 

tiextBLnu»++; 

glTexIiaage2D(GL_TEXTURE_2D, 0, 3, cols, rows, D, Gt*__RGB, GL_UNSIGMED m BYTE, 
texl) ; 

glEndList ( \ ; 

/* new display list*/ 
glNewListtnextDUrmm, GL_COMPILE> r 
t->tex2 - nextDUium; 
nextDLnum++-; 

gl TexI(t*age2D <Gt<_TEXTURE_2 D , D, 3, cols, rows, 0, GL_RGB, GL_UNSIGNED_BYTE, 



PPM^GETR (mapl [ i } ( j ] > t 
PPM_GETR <raap2 [ i ) [ j ] ) ? 



PPHJ3ETG (mapl [i] [ j ] ) ; 
PPMLGETG ( map2 ( i ] [j])? 



PPM_GETB (mapl [ i ]f j ) ) ; 
PPM_GETB ( Jnap2 [ i ] I j ] i ; 



glEndList < I f 

} 



* this will initialize che display lists for the objects 
*/ 

void initialize.objectsUnt argc. char**argvj { 
float tjrp£4J? 

/„* read in the texture */ 
readTextur^Ucef , argv[l], argv{2]f; 

/* create hemisphere */ 

createHenisphered, 50, GUJT R I ANGLS.STR I P) ; 

/* scene */ 
scene. viewangle 
scene. tz = 0; 
scene, ry = 0; 



/* 

* Clear the screen, draw the objects 

* / 

void display O 
{ 

float trap (4 J; 
float height; 

/* clear the screen */ 

glClear (GL_COLOR_BUFFER_BIT ] GLJ>EPTH_BOFFER_BIT ) ; 

/* adjust fcr scene orientation */ 
glMatrixMode <Gl_PROJECTICN> ; 
if (outsideViev) ( 
glLoadlden- ity ( ) ; 

gluPerspec- ive(45 t scene. aspect , 0.1, 1C.0J; 
glTranslatec iQ, C r -3); 
glRotate£<45, 1, 0, 0); 
glRotatefU5 # 0, 1, 0J; 
gl Disable 1GLJTEXTUFE_2D) ; 
glColt>r3t (. 0, .8, .fl); 
] else { 

glLoadldentity ( I ; 

g lu Per spec tive* scene. viewangle, scene. aspect, C.i, 10. 
glTranslatef (0, C, scene, tz I; 
glRotate£< scene. ry, 0, l r 01; 

J 

/* draw our models */ 
glKatrixHodfi !'3L_M0DELV1EKJ ; 
gLPushMacrix t \ ; 

if (outsideview) ( 

/* trans i Or m to where the camera would be */ 
glPushMatrixi) ; 

/* draw a cube Eor the camera */ 
glLoad Identity ( ) ; 
glRotatef (180, I, 0, 0) f 
glTrans} atef l D, 0, scene. ts); 
tapplOl = tmp[l] = tmp[2) = .8; 
tmpf3J = I; 



= 130; 

Fig. 9C 



glMaterialfv<GL_FRONTJ_UD_BACK, GL_.SPECULAR , tmp} ; 
glMaterialf (GL_FR0NT_AND_8ACK, QL_$HttmiES$, 0,0) t 
glMaterial£v«2LJ?R08T„ANDJBACK, G L_AMB I £NT_A*J D_DI FFtfS E f trap); 
glut Sol idCube t-lK- 

/* draw a cone far the view trust rum */ 

glLoadldentity O ; 

height * 1 - scene. tz; 

glRotatee(45, 0, 0, 1> ? 

glTranslatef <Q, Q, -1); 

tmp[0] ■ tmpUJ - 1/ 

trap [2] = 0? 

giHaterial fv«5L_FR0NT„AND,BACK < GL^SPECCJLAR, tiop> ; 

gl Materials (GL_FRONT_AM>_JiACK, G__SHININESS, 0.0) { 

glMaterialfv<GLJPRONT_AN0_BACK. GL„AHBIEKT_ANSLDIFFEJSE, tmp) 

glutSolidConettan (scene. viewangle * 3.14 / 360.0) * height, height , 20, 

glPopMafcrix(1 t 

g 1 En afo 1 e { GL_TEXTCJRE__2 D ) t 



/* now draw the seraisphere *) 
if (lights} { 

tmplO] = tropin =: tmpi2] 

tmp{3] * .87 

g iMa terial Ev \ GL_FR0NT_AND_BACK , GL„SPECULAR , tmp) 1 

g IMaterial E {G L_.FRONT_AND_JBACK , <2__SHINrNESS t 10,0); 

gXH& terial fv*GL_.FRONT_AND_BACK, GL_AWBIEKT_.AND_.Dl FFUSE, trap); 



glCallList lcurrent_texture->texl> 1 
glCallList (HEMISPHERES ; 

if flights) < 

tmplO] = trapltl = tmp!2) - ,5; 
tmp[3] * .5? 

glMaterialEv(GL_FRCHSn_AND_BACK, GL_SPECU'_AR, trap); 
g lHa terial f (GL_FRONT_AND_RACK, G__.SHININESS , 10 . 0 ) ; 
glMaterial £v (GL_FRONT__AND_BACK t GL_AMBrENT_AND„DIFFUSE, tmp); 

} 

glRotatef USO.O, 0.0, 0.0- 1.01; 
glCallList (current_te3tture~>tex2 > ? 
glCallI>i$t (ftEMISPHERE) ; 
glPopMatrixO ; 

fprintf Jstderr, *%sVn", gluErrorStringlglGetErrortU}; 
alutSwapB_£f ers {) ; 

1 

/* 

* Handle Menus 
*/ 

#define m_quit l 
void Select Unt value) 
I 

switch (value] { 
case M_0UJT: 

exit id) ; 

break r 

} 

glut Post Redisplay ( J ; 

} 

void create_menu(' f 

fprintf fstderr r "Press ? for help\n"j; 



glut CreateMenu* Select) ; 
glxitAtadMenuEntryCQuir-, M_QUITU 
g lut At t SchKenu < GL UT_RIGHT_3 uttcn ) t 

} 



/* initializes hading model */ 
void mylnit (void) 

g lEnabl e ( GL„DEPTH_TEST) ; Fig* 9E 

glShadeModel fGL^SMOOTH) ; 

/* texture stuff */ 

glPixelStorei <gl_unpXkj\lignment, si zeof {Guxbytei } ,* 
glTaxPararaeterf {GL_TEXTUR£_.2D t GL^T£XTUre_JWRAP_S, GL_CLAMP> ; 
glTexPararaeterf {GL_JTEXTtfRE_2D, GLu.TEXTUBE_WRAP_T, GU_CLAMP> ; 
gITexParaineterf JGL.„TEXTURE_2D. GL_TEXTtJRE_MAG_FILTER, GL_NEAREST) ; 
glTexParanteterf {G£<_TEXTUR£_2£>, GI^TEXTORE.MIN.FIUER, GJU,N£AR£ST) ; 
g ITexEnv f < GL_TEXTURE_ENV , GL„TEXTURE_EtiV_MODE , GL_DBCAL > ; 
gl Enable {GL_TEXTURE_2D} ; 

) 

/* 

* Called when the windcw is first opened and whenever 

* the window is reconfigured (moved or resized). 
*/ 

void rayReshape ( int w, int h> 
{ 

glViewport (Q r 0, w, hi? /* define the viei/port V 

scene. aspect: = 1 . 0* {CLf loat )w/ ICLt loatlh; 
glMatrixMode(GL_PROJECTION) ; 
glL-oadldentityn r 

gluperspective{ scene. viewangle, scene. aspect , 0.1» 10.0); 
glMultMacrixf (scene. natrix) ; 

glMatri>cMode (Gk„_MOD£LVl£W} ; f* back to model view ma trix */ 

} 

/* 

* Keyboard handler 
*/ 

void 

Key (unsigned char key. ist x, int y) 

{ 

float matrix EL6] ; 

glHatri*Mode (GL^MODE-VlEW) ; 

g lGet F lc*atv ( Gl_K0DEL?I Ety^MATF IK, matrix) ; 

glLoadrdentity ( } ; 

tptiatt fst<terr. "*d - %c ™, key, key) / 
switch {key J [ 
case 'o* : 

if ( louts LdeView { 

f print f (stderr, -outside on "); 
outsideView = l f 

/* turn on bLending V 
gl Enable (GL_3LE>JD) f 

glBlendFunc (SL__skc_ALFKA, GL^OOEjf I MJS^SRCALPHA ) f 
/* We want t:> see color */ 

glTexEnvf { GLJTEXTU RE_ENV , GU.TEXTURE_ENV_MODE, GL_MODUtATEl ; 

/* turn on air spotlight */ 
glEnabIe(GL_LrGKTl) j 

gl Light £v<GI^_UGKTl # GL^AMBIENT, lightAnb) ; 



glLightfviGLJLIGHTl, GI^DIFFUSE, HghtDiff); 
glLightfvtGLJLIGHTl. GE^SPECUtAR , HghtSpec}; 
glL i ght f v (GL^LIGHT 1 , GL_SPOT_D IRSCT ION # 1 ightDir } 
} else ( 

fprintf (stderr, "outside off 
outsideview - 0; 

glTes<Env£ (GLu_TEXTORE„ENV, GL_JPEXTUR E_ENV„M0DE 1 , GU-DECAL) ; 

3 

break; 
esse ' F* : 

fprintf (stderr, "flat "1; 
glShadelfcdel <GL_FLAT) ; 

breaki Fig, 9F 

esse *£'; 

fprintf (stderr, "smooth 

glShadeModel <GL„SMOOTH> i 

break; 
case *y ' : 

printf{ w ry = %£\n H , scerie.ry); 

scene. ry -= 5; 

break; 
case *Y ' : 

scene. ry += 5; 

break ; 
case *z' i 

scene. tz -= ,02; 

fprintf (stderr, " tz - %f *, scene, tz) ; 
break? 
case *Z' i 

scene. tz += .02; 

fprintf (stderr, ■ tz = %f scene.tz); 
break; 
case 'a': 

scene.viewangie -- 1; 

fprintf (stderr, " angle: &£ scene, views nsfleJ ; 
break? 
case * V ; 

scene. vi^wangle += lr 

fprintf Utderr, "angle: %f scene. v i ^wangle) t 

break ; 
case 55: 

glRctat;et(-5, CO, 0,0, i.g); 

break; 
case 57: 

glRotateE(5 f 0.0, 0,0, 1,0); 

break; 
case 52; 

glRotatetc-S, 0.0, 1.0, 0.0) ; 

break; 
case 54: 

glRotatet(5, 0.0, 1.0. 0.0}; 

break; 
case 56: 

glRotatet(5, 1,0, 0.0, 0.0); 

break; 
case 50: 

glRotatef /-5, 1.0, 0.0* 9. in- 
break ; 
case 'q*: 

if (lights} ( 

glDi^abletGL_LlGHT0} ; 
gU>isable(GL_L.IGHTlNG> ; 
lights - Or 

fprintf (stderr, "no lights 



) else { 

glEnable TG ^LIGHTING) ; 
glEnable(GJk_LIGHTO) t 

glLightfvIGL_LlGH70. GL.PGSITION, lightPos); 
glLightfv{Gt_LIGHTO ( GL_AMBIENT» lightAmb) ; 
g_Lightfv*GL_.LIGHTD, GL^DIFFUSE, lightDiffi; 
glUghtfv(GL_tiIGH?0, GL„SFECULAR, lightSpec),- 
lights - 1; 

f print £ {stderr, "lights M; 

} 

break r 
case ' t 1 : 

fprintf (stderr, "texture off "); 
glDisable fGL_TEXTURE„2D) ; 

break r Fig* 9G 

esse * T ■ s 

fprintf t stderr, "texture on M; 

g» Enable tGL_TEXTURE_2Di ; 

break; 
case *?': 

fprintf f stderr, "hjkl - rotate current objectVn")? 

fprintf (stderr, "s/S - shrink / grow the object or zotm the sceneUi*),* 

fprintf (stderr, "a/A viewangle\nM ; 

fprintf (stderr, "z/Z camera position\n">; 

fprintf (stderr, "£/F flat smoothAn"); 

fprintf (stderr, ■•Escape Quits 

break; 

case 27: /* Esc will quit */ 

exit U) ; 

break ; 
default: 

fprintf f stderr, "Unbound key - %d key! ; 
break; 

i 

fprintf (stderr, *\n fc i; 
glMultMatrixi (matrix) ; 
g 2 ut Post Redisplay (J ; 



Main Loop 

Open window with initial window size, title bar, 
RGBA display mode, and handle input events, 

mainUnt argc, char** argvi 

glutlnit f&argc, argv) ; 

glutlnitDisplayMode (GLUT_DOUBLE I GLUT._F.GBA) ; 
parent = glutCreatewindow (argvfOU; 
mylnit O; 

glutKeyboardFunc (Key} ; 
glatReshapeFunc (/nyReshape) ; 
glutei splayFunc (display } j 
cjeate_meni: ( ) ; 

initiali2e„objects{argc, argv*,- 
glutMainLoop( ) ; 



Hfdef WINDOWS 

# include <windows-h> 

flencif 

H include <GL/gl.h> 

* include <GWglut.h> Fig. 10A 

I include "warp.h* 1 
{(include <stdio.h> 
i ** 

* Triangulate a hemisphere and texture coordinates. 

* listNura - display list number 

* r.uisPts - nunber of points to a side 
+ return the display list 

*/ 

void cr eat eHemi sphere lint listNura, int nui&Pts, mt geom) ( 
double incr = 1.0 / nurnPts; 
double u, v, x, y. 2: 
float tx, t2; 
mt i, j; 

/* start the display list */ 

glWewLisc £ 1 : sr Num. Gk_COMPILE_AND_EXECUTE) ; 

/* create the coordinates */ 

/* use the square to circle map */ 

/* across then down */ 

v = 0; 

for a 0; j < numPts ; j++} { 
/* start the tri strip */ 
gl Begin !geom* ; 
u = 0; 

£or (i = 0; i <= numPts; L++J ( 
/* do the top point */ 
/* get the XY2 coords */ 
maplu, v, fcx r &y, &z) t 

/* create the texture coord */ 
tx - x / 2 + ,5; 
tz e z / 2 + ,5; 

if [tx > 1.0 II tz > 1.0 11 tx < 0.0 I) tZ < 0.0) 
printf ("net in range %f %£Vn", tx, tz) ; 

} 

g} ?&xCDord2f (tx, tz) ; 

/* nerval */ 
glNormal3£ (x, y, z)t 

/* create the coord V 
gLVertex3f <x, y, z)- t 

/* Set the XYZ coords */ 
raap(u, v + incr, &x, fry, &2); 

/* create the texture coord */ 
tx * x / 2 + .5; 
tz = z f 2 + .5; 

if ttx •> l.c II tz > 1.0 11 tx < 0.0 || tz < 0-0) 
print! ("net in range %£ %f\pi", tx, tz\ ; 

) 

glTexCoord2f ftx, tz) ; 

/* normal v 
glNormai3i tx, y, 2); 

/* create the ccord */ 



glVertexSf (x, y f z) ; 

/* adjust u V 
u t- incrj 

} 

/* done with the list */ 
glEndO; 

/* adjust v */ 
v *■= incr; 

) 

/* all done with the list *7 
gjEndList O i 

} 



Fig. 10B 
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Filing Date 



Group Art Unit 



Examiner Name 
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As a below named inventor, I hereby declare that: 

My residence, post office address, and citizenship are as stated below next to my name. 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural 
names are listed below) of the subject matter which is claimed and for which a patent is sought on the invention entitled: 



METHOD FOR INTERACTIVELY VIEWING FULL-SURROUND 
IMAGE DATA AND APPARATUS THEREFOR 



the specification of which 

K is attached hereto 
OR 

□ was filed on (MM/DD/YYYY) 



(We of the Invention) 



as United States Application Number or PCT International 



Application Number 



j and was amended on (MM/DD/YYYY) 



(if applicable). 



I hereby state that I have reviewed and understand the contents of the above identified specification, including the claims, as 
amended by any amendment specifically referred to above. 

I acknowledge the duty to disclose information which is material to patentability as defined in 37 CFR 1 .56. 



I hereby claim foreign priority benefits under 35 U.S.C. 1 19(a)-(d) or 365(b) of any foreign appiication(s) for patent or inventor's 
certificate, or 365(a) of any PCT international application which designated at least one country other than the United States of 
America, listed below and have also identified beiow, by checking the box, any foreign application for patent or inventor's certificate, 
or of any PCT international application having a filing date before that of the application on which priority is claimed. 



Prior Foreign Application 
Number(s) 



Country 



Foreign Filing Date 
(MM/DD/YYYY) 



Priority 
Not Claimed 



Certified Copy Attached? 
YES NO 



□ 
□ 
□ 
□ 



□ 
□ 
□ 
□ 



□ 
□ 
□ 
□ 



I Additional foreign application numbers are listed on a supplemental priority data sheet PTO/SB/02B attached hereto: 



I hereby claim the benefit under 35 U.S.C, 1 19(e) of any United States provisional application(s) listed below. 



Application Number(s) 



60/071,148 



Filing Date (MM/DD/YYYY) 



01/12/98 



I | Additional provisional application 
numbers are listed on a 
supplemental priority data sheet 
PTO/SB/02B attached hereto. 
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DECLARATION — Utility or Design Patent Application 



I hereby claim the benefit under 35 U.S.C. 120 of any United States application (s), or 365(c) of any PCT international application designating the 
United States of America, listed below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior 
United States or PCT International application in the manner provided by the first paragraph of 35 U.S.C. 112, 1 acknowledge the duty to disclose 
information which is material to patentability as defined in 37 CFR 1.56 which became available between the filing date of the prior application 
and the national or PCT international filing date of this application. 



U.S. Parent Application or PCT Parent 
Number 



Parent Filing Date 
(MM/DD/YYYY) 



Parent Patent Number 
(if applicable) 



C] Additional U.S. or PCT international application numbers are listed on a supplemental priority data sheet PTO/SB/02B attached hereto 



As a named inventor, I hereby appoint the following registered practi tioners) to prosecute this app lication and to transact all business in the Patent 



and Trademark Office connected therewith: Q Customer Number 

OR 



Registered practitioners) name/registration number listed below 



Place Customer 
Number Bar Code 
/ atet ftnYr? 



Name 



Registration 
Number 



Name 



Registration 
Number 



Robert A, Westerlund 
Raymond H. J. Powell, Jr. 
Ramon R. Hoch 



31,439 
34,231 
34,108 



I — I Additional registered practitioner(s) named on supplemental Registered Practitioner Information sheet PTO/SB/02C attached hereto, 



Direct all correspondence to: □ Customer Number 

or Bar Code Label 



OR Q5 Correspondence address below 



Name 



Westerlund & Powell, P.C. 



Address 



122 N. Alfred Street 



Address 



City 



Alexandria 



State 



VA 



ZIP 



22314-3011 



Country 



USA 



Telephone 1(703) 706-5862 



Fax 1(703) 706-5860 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and belief are 
believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under 18 U.S.C. 1001 and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 



Name of Sole or First Inventor: 



□ A petition has been filed lor this unsigned inventor 



Given Name (first and middle Hf anvtt 



Family Namp nr Surname 



Ford 



OXAAL 



Inventor's 
Signature 



Date 



Residence: City 



Cohoes 



State 



NY 



Country 



USA 



Citizenship 



US 



Post Office Address 



42 Western Avenue 



Post Office Address 



City 



Cohoes 



State 



NY 



12047 



1 



Country 



USA 



□Additional inventors are being named on the supplemental Additional Inventor(s) sheet(s) PTO/SB/02A attached hereto 
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